Deep Reinforcement learning (applied to CFD)
TD3 (off-policy) & PPO (on-policy) - Validation for DRL algorithm
(a) burgers equation (no control)
(b) TD3-Control
(c) PPO-Control
Description: Machine learning is applied with CFD as a domain. Here, the Burgers equation is simply Controlled by reinforcement learning is shown above.
(Burgers equation is one-dimensional Navier-Stoeks equation - here it is solved by spectral method).
(a) is the burgers equation without control. Initial condition is sinusoidal, you can see that the sin waveform is broken down by the viscous term.
The results of training with reinforcement learning to maintain the initial state (sin waveform) are (b) and (c).
(b) is implemented with the TD3 algorithm of the deterministic policy family,
(c) is implemented as a PPO algorithm of the stochastic policy family.
Turbulence control for drag reduction through deep reinforcement learning
(a) Actor-Critic network structure
(b) No control shear field
(c) Control of shear field
Description: Drag reduction in a fluid is associated with a variety of social problems ranging from economic loss to environmental problems. The goal of this research is to develop a strategy
for drag reduction in turbulent channel flow by controlling blowing suction using Deep reinforcement learning. As a result, performance is increased by approximately 5% over previous deep learning models. My involvement in this work was to construct a reinforcement
learning algorithm directly in Tensorflow, connect it to CFD, and then trained the network
and analyzed it. (a) Actor-Critic network structure inserted in TD3 algorithm, (b) No control shear field (c) As can be seen, the DRL control reduces the high shear stress (RED to BLUE).
and This is currently being worked on in preparation for submission to a journal thesis.